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Experience with Online and Open-Web Exams 

Edward F. Gehringer | Barry W. Peddycord III 

As homework and other aspects of education migrate to a computer-based format, on-paper exams are beginning 
to seem like an anachronism. Online delivery is attractive, but comes with a myriad of implications not apparent 
at first glance. It affects the kinds of questions that can be asked and complicates administration of the exam, but 
it may make grading quicker. An online exam does not necessarily make cheating easier, though it demands new 
approaches to preventing cheating. An open-web exam is a special kind of online exam, in which students are al¬ 
lowed to use the Internet while taking the exam. It has its own set of advantages and disadvantages. For example, 
it provides a more authentic experience, but may make it hard to tell if students have done their own work. The 
authors report on a research study of online exams, based on the observations of hundreds of students and faculty 
who filled out two online surveys. Results will guide instructors in choosing the format that is right for their class. 


Most examinations are administered in an en¬ 
vironment that is becoming artificial. Today, ideas 
are routinely communicated using computers; for 
example to perform technical work or even write a 
piece of prose many use the aid of a computer. That 
is the situation that students are in when they take 
an exam on paper. Faculty have plenty of opportu¬ 
nities to administer tests online. Textbook publish¬ 
ers, as well as learning-management systems, have 
online modules that could be used to deliver quiz¬ 
zes or exams. Students can use their laptops to take 
an exam online, even in an ordinary classroom. But 
online administration raises several issues, which 
instructors should think through before taking the 
plunge. 

To uncover the implications of online exams, 
the authors set up a research study, surveying in¬ 
structors on four teaching-related listservs: the En¬ 
gineering Technology listserv; SIGCSE-members 
(computer science faculty); the College Board’s AP- 
CS list; and the listserv of the Professional & Or¬ 
ganization Development Network in Higher Educa¬ 
tion; and students in the first author’s classes from 


Fall 2009 until Spring 2011. Eighty-five instructors 
and 315 students responded to the survey. All of 
them had experience administering or taking online 
and/or “open-Web” exams, where students were 
given unfettered access to the Internet during the 
exam, but forbidden to communicate with others. 
This study investigated several aspects of online ex¬ 
ams, classifying its observations into six categories: 
material covered, administration of the exam, grad¬ 
ing, academic integrity, handwriting vs. coding, and 
miscellaneous. 

MATERIAL COVERED 

Online exams make it possible to ask more kinds 
of questions. An instructor might have the students 
run a particular simulation or animation and ex¬ 
plain the observed results. The instructor might ask 
the students to peruse a set of online documents to 
find the inputs necessary to solve a certain problem. 
In a traditional hardcopy exam format this would 
waste paper. The instructor might forgo providing 
some useful documents, simply to save paper. 

In many fields, exam questions can often be 
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answered by drawing a diagram. Unfortunately, it 
is difficult to draw diagrams on traditional desk¬ 
top or laptop computers. Thus, the online format 
precludes asking such questions unless all students 
have tablet computers, and the testing software is 
capable of handling such diagrams. 

Another complication is that it may be difficult 
to create multipart questions. The testing software 
may not support multipart questions, and even if it 
does, it may impose restrictions that make multi¬ 
part questions infeasible. In the Moodle learning- 
management system (LMS), for example, a manu¬ 
ally graded “essay” question is only allowed to have 
one part. It’s not possible to get around this restric¬ 
tion by treating the parts of the question as separate 
questions, because most testing systems allow the 
instructor to “randomize” the order of questions to 
inhibit cheating. After randomization, the different 
questions would be distributed throughout the exam. 

ADMINISTRATION 

If timed, online exams afford all students the 
same amount of time. No one gets the test paper be¬ 
fore another; no one can try the instructor’s patience 
by staying late. Online exams allow time-shifting: 
students can be allowed to begin the exam at a time 
of their choosing. When exams are administered 
outside of class, students can be given almost an un¬ 
limited amount of time. But time-shifting comes at 
a cost. It makes it difficult or impossible to proctor 
the exam, and thus enables cheating. 

Online exams also allow location-shifting. Stu¬ 
dents can be permitted to take exams off site while 
on business travel, deployed in the military, or out 
of town for a family emergency. Location-shifting 
can also expedite misconduct. Traditionally, an 
instructor can abate cheating by bringing all stu¬ 
dents together in the same room, where they can 
be observed. It may still be possible to allow a few 
students to take exams out of class in exceptional 
cases. These students’ answers can be subjected to 
stricter scrutiny. For example, their scores can be 
compared with other exams taken face-to-face by 
the same students. 

Online exams have other drawbacks. Giving 
an exam to an unregistered student is much more 
difficult; such a student may be finishing up an in¬ 
complete in a course, or taking an exam with a dif¬ 
ferent section because of schedule constraints. With 
a traditional paper-and-pencil exam, the student can 


simply be handed a test paper in class. With an 
online exam, the student either needs to be added 
to the class roster used by the testing system, or a 
separate exam may have to be created in the testing 
system, for just this student. 

With an online exam, innocent mistakes can be¬ 
come security risks. Many an instructor has inad¬ 
vertently set an exam to turn on too early, or made 
answers available to students before the exam ended. 
Such mistakes may go unnoticed for a while, espe¬ 
cially if students are taking the exam at times of 
their own choosing. Exams do not look quite the 
same in all browsers. Text boxes may appear in dif¬ 
ferent places. In the Moodle quizzing system, we 
found that text boxes covered up the questions on 
certain browsers. Early versions of Google Chrome 
tended to freeze up, especially when the window 
was scrolled horizontally. Sometimes this forced us 
to allow students to retake the exam. 

In some networks, a connection may be closed 
if a student spends “too much” time working on a 
problem without touching the computer. One of our 
instructor respondents reported that a student was 
locked out of an unfinished exam after the connec¬ 
tion was dropped, and ultimately needed to retake 
the entire exam. Some less-than-honest students 
have used the network as an excuse for not finishing 
an exam, reporting a network problem but failing to 
show any proof.. 

There are also problems related to saving work. 
In Moodle, when time expires, the last saved copy 
of a student’s work is submitted, and any unsaved 
changes are lost. Inevitably this catches some stu¬ 
dents on their first online exam. One instructor us¬ 
ing Blackboard reported that students who click on 
“Save” instead of “Submit” at the end of the exam 
lose their work. In some systems, such as Desire- 
2Learn, the Save and Submit buttons are near each 
other, raising the danger of unintentionally termi¬ 
nating the exam. 

GRADING 

Instructors who teach large classes may be attract¬ 
ed to online exams because of the potential for auto¬ 
mated grading. When it works, it is a godsend. On 
the downside, it tends to encourage multiple-choice 
questions, since they can be reliably auto graded. This 
may not be the best approach pedagogically, but it is 
not a consequence of online administration; the same 
tendency occurs with paper-based exams. 
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Most systems can also automatically grade 
matching and numeric-answer questions. The time 
savings allows more frequent exams, which can 
produce learning gains (McDaniel, Roediger, & 
McDermott, 2007). Moreover, feedback can be im¬ 
mediate, and the student is likely to pay more atten¬ 
tion to it while the question is still in mind. In the 
case of final exams, online feedback benefits stu¬ 
dents who must leave town before their exams are 
graded. They can log in, see where they lost points, 
and challenge suspected grading errors. 

Online systems allow questions that have text 
answers, which must be manually graded. Manu¬ 
al grading is often easier on an online exam. In a 
paper exam, when many students make the same 
mistake, instructors tire of writing identical feed¬ 
back on each paper. With an online system, they 
can save a set of standard comments, then cut and 
paste them into different exams. If a wrong answer 
is given to a multiple-choice question, an automated 
system can give feedback to the student that is ex¬ 
plicitly tailored for the specific misconception be¬ 
hind each incorrect response. 

With online exams, no paper-shuffling is in¬ 
volved in manually grading questions. An instruc¬ 
tor can finish one question on every student’s exam 
before moving on to the next question. This makes 
it easy to construct and apply a mental rubric, and 
promotes consistent grading of essay questions. It 
may also save time, because the instructor does not 
lose context in switching from question to question. 

In a large class, the exam papers do not have to 
be divvied up among multiple graders; several peo¬ 
ple can grade simultaneously. If someone forgets to 
grade a question, the instructor can point that out, 
and the grader in charge of that question can handle 
it without the need to retrieve the exam paper from 
whoever now has it. 

Online systems automatically record grades. 
This prevents accidental loss of information. It also 
provides valuable data for assessment, especially if 
individual questions can be tied to learning objec¬ 
tives. 

But there are downsides to automated grading. 
For single-word and fill-in-the-blank answers, it is 
nearly impossible to anticipate all of the spellings, 
punctuation, and wordings that students may give. 
If the answer is numeric, precision and rounding 
can cause problems. In some systems, an instructor 
cannot review grades assigned by the system, except 


by navigating to each student’s exam, and clicking 
on the question number. Thus, automatic grading 
can sometimes take more time than manual grad¬ 
ing. It is also more difficult to give partial credit on 
automatically graded questions; in fact, some sys¬ 
tems may not allow it at all. This can negatively 
impact student scores. 

ACADEMIC INTEGRITY 

Perhaps no topic attracted as much attention 
from instructors as academic integrity. In many of 
their minds, online testing is associated with exam¬ 
inations that are not proctored, which offer copious 
opportunities for collusion. It is, of course, possible 
to proctor online exams, especially in classrooms at 
schools that require students to own laptops. Soft¬ 
ware such as SecurExam or ExamSoft can “lock 
down” browsers to prevent students from commu¬ 
nicating or visiting unauthorized sites during the 
exam. Most exam applications can randomize the 
order of questions, and the order of multiple-choice 
answers, which makes it harder for students to copy 
each other’s work. Numerical questions can be set 
to give students different input values, so that the 
correct answer will be different for each student. 
The exam can even be set up to prevent students 
from revisiting earlier questions, though many stu¬ 
dents find that frustrating. 

HANDWRITING VS. CODING 

Handwriting is a handicap for both students and 
instructors. Most students can type faster than they 
can write, and the ability to edit, spell-check, and 
grammar-check what they have written helps them 
produce better prose. They can spend more time 
thinking, and less time writing. Faculty do not lose 
time trying to decipher student handwriting. 

Offsetting these advantages is the need to code 
questions for the testing software. Fill-in-the-blank 
and short-answer questions are hard to code, be¬ 
cause of the difficulty of anticipating all correct 
responses. Multiple-choice is easy to code, but it 
is time-consuming to devise distractor responses 
that mimic student misconceptions. If a question 
requires complicated formatting or involves filling 
in blanks in a table, it generally cannot be created 
with a WYSIWYG editor in the testing applica¬ 
tion. It is necessary to use an external application, 
such as Dreamweaver or Microsoft Expression Web 
Designer, but these do not support the commands 
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needed for automatic grading. A teacher must go 
back and forth between the applications, usually 
several times, to eliminate all formatting glitches. 

Beyond that, the testing system may treat cer¬ 
tain characters as control codes. For example, the 
Moodle quizzing system treats a “<” as the begin¬ 
ning of an HTML tag, even when it appears in an 
answer blank. When it saves the answer, it drops 
everything between the “<” and the “matching” “>”, 
which can cause equations and programming-lan¬ 
guage statements to be completely discarded. 

Mathematical questions require a specialized 
form of data entry, such as an equation editor. Stu¬ 
dents and instructors are rarely accustomed to these, 
and may find them hard to use. Moreover, students, 
who on a paper exam might show their work, are 
reluctant to type each step into an equation editor. 
So there is less information to use in awarding par¬ 
tial credit. 

MISCELLANEOUS 

Online exams may increase privacy, since stu¬ 
dents do not have a chance to see others’ scores 
when papers are returned in class. Online exams 
save paper, which is not only eco-friendly, but also 
a boon to tight supply budgets. It is convenient for 
an instructor to reuse questions that have been used 
in an earlier online exam, but be aware that there 
are Web sites like Course Hero that collect previous 
exams and solutions from students. This is critical 
if students will have Web access during the exam. 

Unfortunately, it is harder for students to scan 
through an online exam before starting work on it. 
Accessibility may be an issue for the visually im¬ 
paired, especially if the exam uses material outside 
the testing system, such as animations. Finally, if 
an instructor has several years’ worth of questions 
stored in a particular online application, and the in¬ 
stitution switches to a different system, it may be 
very time consuming and difficult to convert the 
questions. 

ISSUES SPECIFIC TO OPEN-WEB EXAMS 

An exam may be administered on a computer, 
or on paper. Students may or may not be allowed to 
browse the Web. The two issues are independent; 
one can give an open-Web exam where students 
write their answers on a test paper. Open-Web on- 
paper exams bypass all of the technological hurdles 
enumerated in previous sections. 


Whether delivered by computer or not, open- 
Web exams have their peculiar advantages and dis¬ 
advantages. Because the environment resembles 
the setting in which students and others perform 
their daily work, the exam can be a more authen¬ 
tic assessment (Wiggins, 1990) of what the students 
know. Questions can be posed that require the stu¬ 
dents to search for and apply information. Recall 
questions, however, cannot be asked. Even explana¬ 
tions can be looked up, so if students are asked to 
tell why a particular phenomenon occurs, instruc¬ 
tors should check that they have not simply cut and 
pasted their answer. Even if they have not copied an 
answer, they may have reworded one. One needs to 
frame questions carefully in order to judge whether 
students really do understand the concept. 

After their first open-Web exam, several stu¬ 
dents volunteered their opinion that they would 
have done better had they not spent so much time 
looking up information on the Web. This is consis¬ 
tent with the observations of Boniface (1985) and 
Ioannidou (1997), as quoted by Rakes (2008), who 
found that some students performed more poorly on 
open-book than on closed-book tests, because they 
spent time looking through their textbook or notes. 
On subsequent exams, the author warned students 
to spend less time browsing and more time answer¬ 
ing the questions. 

Open-Web exams close certain avenues for 
cheating while opening others. In most open-Web 
exams, it is reasonable to allow the students also 
to consult print resources during the exam. This 
removes the need to monitor what the students are 
reading during the exam. Unauthorized commu¬ 
nication is another matter. There is no technologi¬ 
cal way to prevent students from consulting others 
during the exam. Even if all known e-mail, instant 
message (IM), and other communication programs 
were blocked, students could write their own chat 
server and run it on a Web site during the exam. 
Locking down the browser simply defeats the pur¬ 
pose of an open-Web exam. 

Thus, vigilance is necessary on the part of the 
proctors. Fortunately, this is not as difficult as it 
might seem. A proctor who sits in the back row of 
a theater-style classroom can easily see twenty to 
thirty laptop screens at one time. It is more difficult 
in a classroom with a level floor, but these class¬ 
rooms typically hold far fewer students. Being be¬ 
hind the students is a deterrence, because a student 
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Table 1: Results of the student survey on online and open-Web exams 



All students 

Open-Web online 

Open-Web on paper 

Number responding 

315 

234 

81 

The questions were 
relevant to the material 
covered in class. 

3.89 

4.07 

3.37 

The test format of open- 
book, open-web was beneficial. 

4.29 

4.31 

4.25 

The format was relevant 
to the way work is done in 
business/profession. 

3.72 

3.87 

3.30 

Taking the test online (or 
open-Web) worked well 

3.58 

3.53 

3.73 

The exam structure 
allowed students to cheat. 

1.91 

1.91 

1.93 


Note: Students were quite positive on the format of the exam, especially on the ability to use the Web. The classes that wrote 
their answers on paper thought the format worked slightly better than the classes that took the exam online. This may reflect 
the difficulty that students in the latter group had with the software (e.g., losing unsaved work and having some answers truncated). 


can never know whom the proctor is watching. 

In the first published study on open-Web ex¬ 
ams, Williams and Wong (2007) found that stu¬ 
dents were only slightly more likely to believe that 
the exam structure allowed students to cheat (2.83 
for open-Web exams, vs. 2.63 for closed-book ex¬ 
ams on a Likert scale of 1 to 5, with 5 meaning 
cheating was easy, n = 54). Their exams, however, 
were taken at different locations and times. In our 
classes, students were required to take the exam in 
designated rooms at designated times. Our student 
respondents (n = 315) gave an average score of only 
1.91 to the question of whether students were able 
to cheat. Table 1 reports the results of our student 
survey. Gehringer (2010) goes into more detail on 
the 2009 results. 

THE BIG PICTURE: RECAPPING OUR FINDINGS 

Table 2 summarizes the advantages and disad¬ 
vantages of online and open-Web formats. We may 
observe that the inherent disadvantages of online 


and open-Web formats are largely related to aca¬ 
demic integrity. Even the two inherent “adminis¬ 
trative” disadvantages relate to the difficulty of de¬ 
tecting cheating. Over time, then, we may expect 
most categories of disadvantages to disappear. The 
decision of whether to test online will then depend 
heavily on what kinds of cheating the instructor 
wishes to preclude. 

The author’s students were quite positive on the 
open-Web format. To the question, “The test for¬ 
mat of open-book, open-Web was beneficial,” they 
gave a rating of 4.29 (on a scale of 5). On the ques¬ 
tion, “Taking the test as an open-Web exam worked 
well,” the students who used the paper-based exam 
rated it slightly better (3.73, n = 81) than the students 
who took their exam online (3.53, n = 234). This 
may reflect the difficulties that some students had 
with the online-exam software. 

Open-Web exams have a particular advantage 
in situations where students need to look up large 
amounts of material during the exam. Online exams 
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Table 2: Advantages and Disadvantages of Online and Open-Web Exams 

Disadvantages may be either technological (-) or inherent (x) 

COVERAGE 


+ 

Can ask questions that require a lot of background material 

- 

Cannot ask questions that require a diagram for an answer 

- 

Difficulty of mixing automatically graded and manually graded questions 

ADMINISTRATION 

+ 

All students have the same amount of time 

+ 

Students can take the exam at different times 

+ 

Students can take the exam in different places 

- 

It is more difficult to give an exam to a student not registered for the class, or one 
who needs to take the exam early 

- 

Some browsers may have trouble with the exam, or with certain questions 

- 

Students may fail to save their work, or accidentally exit the exam 

- 

Students in high-security environments where only verified software/Web sites are 
allowed may not be able to access the exam 

- 

Network problems may abort exam attempts 

X 

Time-shifting and location-shifting facilitates cheating 

X 

Students may use fabricated network problems as an excuse for their own lack of preparation 

GRADING 

+ 

Automatic grading may save time 

+ 

Allows exams to be more frequent & shorter 

+ 

Automatic grading perceived as more objective 

+ 

Immediate feedback on automatically graded questions 

+ 

Students have more of an opportunity to look over graded final exams 

+ 

Students can quickly see how they compare with peers 

+ 

Grading a single question at a time may be more efficient 

+ 

Easier for multiple graders to interact 

+ 

Grades are automatically recorded 

+ 

More data is available for assessment and accreditation 

- 

Automatic grading may waste time 

- 

Harder to give partial credit, especially on mathematical questions. 

- 

Rounding may be tricky to handle 

- 

If an external gradebook is used, export/import of grades may be tricky 
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ACADEMIC INTEGRITY 

+ 

Can randomize questions and answers to inhibit cheating 

+ 

Can use different data sets to inhibit cheating 

+ 

To discourage communication, exams can be proctored 

+ 

To inhibit cheating, can prevent students from going back to earlier questions 

+ 

Browsers can be “locked down” to prevent communication 

- 

Randomization does not work well on all systems 

X 

Easier for students to save copies of all questions 

X 

Proctoring not possible for students who take exam at a time or place of their choosing 

X 

Students find it frustrating if they cannot go back to earlier questions 

HANDWRITING YS. CODING 

+ 

Easier for students to type than write 

+ 

Easier for faculty to read typing than handwriting 

- 

Automatic grading of short-answer qq. is error prone and must be checked manually 

- 

System may discard answers that contain arbitrary special characters 

- 

Equation editors are hard for students & instructors to use 

- 

Screen layout of questions may be unreadable on some browsers 

X 

Time consuming to code questions correctly 

MISCELLANEOUS 

+ 

Some students find using a computer to be less stressful 

+ 

Privacy-students have less chance to see others’ graded exams 

+ 

Easier to reuse questions 

+ 

More “environmentally friendly” 

- 

Accessibility-students with ADHD or RSI may be disadvantaged 

- 

Harder to get to know students’ names without passing back exams 

- 

Harder to scan through test at a glance 

X 

Some students find using a computer to be more stressful 

X 

If institution switches to new system, may be difficult to retrieve and use old questions 

OPEN-WEB 

+ 

Closer to authentic assessment 

+ 

Cannot cheat by using unauthorized materials 

+ 

Allows more research & application questions 

X 

Hard to tell whether students understand the answer or have simply cut & pasted it 

X 

Students may waste time browsing Web for an answer 

X 

Easier to cheat by communicating with others 

X 

Locking down browsers defeats the purpose of an open-Web exam 
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are beneficial in a wide range of situations, from 
distance-ed classes to classes where data is being col¬ 
lected for accreditation (Table 3). Open-Web exams 
have no obvious disadvantages except academic 
integrity. But even this is debatable, because the 
ability to give students different versions of the 
same question can make cheating harder. Online 
exams have some technological disadvantages, 
relating to the difficulty of entering diagrams and 
equations (though this is being mitigated by tablet 
computers). 


Table 3: Situations where Online (ol) and 

Open-Web (ow) Exams Help and Hurt 

Features of classes where online exams work well 

• (ol) Distance-ed classes 

• (ol, ow) Classes with technologically astute 
students 

• (ol) Classes where randomizing inputs can 
give each student a different problem 

to work 

• (ol) When simulations need to be run 
and interpreted 

• (ow) Classes where large amounts of 
material have to be looked up 

• (ol) Classes where data needs to be 
collected for accreditation 

• (ol) Large classes 

Features where online exams do not work well 

• (ol) Material where one typically needs to 
draw diagrams or pictures as evidence of 
understanding 

• (ol) Material where one has to show the 
work on a problem, especially when the 
work involves symbolic equations (calculus, etc?) 


CONCLUSION 

Online and open-Web exams have many advan¬ 
tages. Students seem to prefer them, and they ap¬ 
pear to allow students to better demonstrate what 
they know. The strength of online exams is auto¬ 
mated grading, but it comes with an overhead that 
may be difficult to justify in a small class. Small 
classes may do better with open-Web exams admin¬ 
istered on paper. Although academic integrity is a 
major concern in either format, the evidence seems 
to indicate that it can be successfully addressed. 
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